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The generalized entropic measure, which is optimized by a given arbitrary distribution 
under the constraints on normalization of the distribution and the finite ordinary 
expectation value of a physical random quantity, is considered and its Lesche stability 
property (that is different from thermodynamic stability) is examined. A general 
condition, under which the generalized entropy becomes stable, is derived. Examples 
known in the literature, including the entropy for the stretched-exponential distribution, 
the quantum-group entropy, and the ?c -entropy are discussed. 
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I. INTRODUCTION 



There is great diversity in statistical distributions observed in nature. This is 
apparently a challenge for traditional statistical mechanics. In view of traditional 
statistical mechanics based on Boltzmann-Gibbs-Shannon entropy, a significant number 
of distributions observed in complex systems are actually anomalous. Examples include 
granular materials, glassy systems, self-gravitating systems, biological systems, and 
seismicity. An important point here is that these anomalous distributions can persist for 
very long periods of time, much longer than typical time scales of underlying 
microscopic dynamics. This fact naturally leads to a question if there would be a 
framework for understanding such diverse statistical phenomena in a unified manner. In 
this respect, the principle of maximum entropy pioneered by Gibbs and Jaynes may be 
though of a one such [1]. Then, if one wishes to describe such anomalous distributions 
based on the principle of maximum entropy, there seem to be only two ways to be 
addressed. One is to modify the form of the constraints, and the other is to generalize the 
Boltzmann-Gibbs-Shannon entropy. The latter is the standing point that we take in the 
present work. 

In a recent paper [2], one of the present authors has presented an entropy generating 
algorithm and has constructed a very general class of entropic measures which are 
optimized by given distributions under the appropriate constraints on normalization and 
the ordinary expectation value of a physical random variable such as the energy. In spite 
of its mathematical consistency of the discussion, however, it is still not clear if the entire 
class of such generalized entropies may behave as good measures. 
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In this paper, we examine the concept of stability proposed by Lesche in Ref. [3] (see 
also Ref. [4]), which should be satisfied by any physical entropic measure. We shall 
derive a general condition, under which the generalized entropy can satisfy the Lesche 
stability property (that is different from thermodynamic stability). The stability properties 
of a number of examples including the entropy for the stretched-exponential distributions 
[5], the quantum-group entropy [6], and the k: -entropy [7-10] are also discussed. 



In Ref. [2], an algorithm has been presented for generating a generalized entropy 
which is optimized by a given arbitrary distribution under the constraints on 
normalization of the distribution and the ordinary expectation value of a physical quantity, 
(2J(=i,2, of interest (e.g., the internal energy), where W is the number of 
microscopically accessible states. 

Given a normalized distribution = /(« + ;S2;)},^j 2, ,w" the corresponding 
generalized entropy optimized by it is constructed as follows: 



Here, a and P are the Lagrange multipliers associated with the constraints on 
normalization and the ordinary expectation value of {2,},=i,2, respectively. A[p;t) is 
a quantity given by 



II. GENERALIZED ENTROPY 




(1) 
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A[p;t) = Y,(p,-f(t))^ 



(2) 



with the notation 



(x)+ = max{0, x}. 



(3) 



c is the constant which should be detennined in such a way that S[p] vanishes for the 
completely ordered state, = p."^ = S^j (1 < j < W). f(t) is a function that determines 
the form of the distribution, p.. For the sake of simplicity, this function is assumed to be 
a monotonically-decreasing function with the range (0, 1) and to satisfy the condition 



• max 

I dtfit) 



< oo. 



(4) 



where t^^) is the domain of f(t), i.e., f(t) ^ 1 (0) as ? ^ t^^ (t^^). 



It can be seen that Eq. (1) is written in the following form: 



S[p] = t 



(=1 



Pif-\Pi)- j dtm 

f~\0) 



+ j dtfit)-/-' (I), 
f~Ho) 



(5) 



where / ^ is the inverse function off. Moreover, if / Ms piecewise differentiable, as 
assumed here and hereafter, then S[p] can be further rewritten as follows [11]: 
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s[p]=Y, \ dtf-\t)-\dtf-\t). 

! = 1 



(6) 



With this form, it is now evident that the stationarity condition on the functional, i.e., 
^{s[p]~fX^^_^Pi~P^J_^PiQi^ = ^' ill fact yields the optimal distribution 

{p, = /(a + iS(2,)},..,,2,.,^. 

The construction in Eq. (1) with Eq. (2) is a general mathematical procedure for 
generating a concave functional of {p,},=i,2, ,iv addition, as shown in Ref. [2], S[p] 

satisfies the H-theorem for the master equation combined with the principle of 
microscopic reversibility. 



III. STABILITY CRITERION 



It is not expected that whole class of generalized entropic measures expressed in the 
form in Eq. (1) or Eq. (6) are physically relevant, even though they are concave and 
satisfy the H-theorem. In order for an entropic measure to be experimentally robust, it is 
necessary for the measure to satisfy the stability condition proposed in Ref. [3]. This 
concept is stated as follows. Usually, what is experimentally measured is not directly a 
statistical entropy, L, itself but a distribution. Repeating the same experiment, an 
experimentalist will obtain a distribution, which may be slightly different from the 
previously obtained one. If L is of physical relevance, then at least its value should not 
change drastically for two slightly different distributions, {p,},=i,2,....,w iP' Jw=i,2, ,w 
Mathematically, this implies 
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(V£>0) (35 >0) 



p- p'\\i<5 



< £ 



(7) 



for any value of W, where IIAIIi= I A, I and E^^^ is the maximum value of L. To 
examine this condition for the quantity in Eq. (1), we analyze the following inequality: 

T ^ max 

\S[p]-S[p']\< \ dt\A[p;t)-A[p';t)\+ j dt\A[p;t)-A[p';t)\, (8) 



where T satisfies 



t^<f-\l/W)<t<t^. 



(9) 



To evaluate the right-hand side of Eq. (8), we notice the following properties: 



\A[p;t)-A[p';t)\<\\p-p'\\„ 



(10) 



\A[p; t) -A[p ■t)\< Wf{t) ( t > f-\l I W)). 



(11) 



Using Eqs. (10) and (1 1) in Eq. (8), we find 



\S{p}-S{p"[\<G{T), 



(12) 
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• max 

G(T) = {T-t^J\\p- p \\,+W \ dt m . (13) 



Eq. (12) holds for any value of T satisfying Eq. (9). Let us take T = Tq which makes 
G(t) minimum: 

To=ri\\p-p'\\i/w). (14) 

Therefore, we have 

\S[p]-S[p']\<{f-'(\\p-p'\\Jw)-t^^}\\p-p'\\, 

^ max 

+ W \ dtfit) 

f-'{\lp-p'lli/w) 



/('„,aJ 



= Wt^^^f(t^J-t^J\p-p'\\,-W j dtf-\t). (15) 



llp-p'lli/W 



Noticing that Eq. (6) takes the following maximum value for the equiprobability 



i/w 



S,.^ = W \ dtf-\t)-\dtf-\t), (16) 



we have 
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S[p]-S[p'] 



<B(\\p-p'\\„W), 



(17) 



1 

Jo 



dtf-\t)-(t^^/W)\\p-p'\\, 



B(\\p-p'\\„W)^ 



f^"" dtf-\t)-(l/W)f^ dtf-\t) 



(18) 



where we have used the fact that tf{t) tends to vanish in the limit ? — > ^^ax' due to Eq. 
(4) as well as the property, /(O— >0 (?— >?max)- Therefore, we conclude that the 
generalized entropy is stable in the thermodynamic limit, — > oo, if 

lim \\m B{\\p-p'\\,,W) = 0. (19) 

This is our main result. Notice that this order of taking the limits is essential for the 
Lesche stability criterion. 

Notice that B {\\p - p' \\^,W) 'm Eq. (18) is an indeterminate form in the limit W — > <». 

A particular case when application of L'Hopital's rule once to this limit is sufficient, then 

we have 



S{p]-S{p'] 



<Cllp-p'lli, 



(20) 
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c = 



/-^(+0)-/-'(l-0) 



(21) 



f-\+o)-Cdtf-\t) 



So, in this case, taking 5 as 5 = £ / C, we see that the generalized entropy satisfies the 
stability condition in Eq. (7). 

Closing this section, we notice that in Ref. [12] the continuity and stability properties 
of a class of generalized entropies are discussed by employing an approach different from 
the present one. 



In this section, we discuss some examples of stable generalized entropies known in the 
literature. 

1. Entropy for stretched-exponential distribution 

In this case, f(t) is taken to be 



where y s (0, 1) and t s {t^, t^^) = (0, qq). Substitution of this function into Eq. (6) 
gives rise to the following generalized entropy [2,5]: 



IV. 



EXAMPLES 



/(0 = exp(-rO, 



(22) 
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55JP] = L r(l + l/7,-lnp,)-r(l + l/7), (23) 



1=1 



where r(u, x) is for the incomplete gamma function of the second kind, r(u, x) = 

poo 

dt t"~ e~' , and r(u) = r(u,0) is the ordinary gamma function. Since 
f-\t) = (-InO'"' with t € (0, 1), C in Eq. (21) is calculated to be 



C=lim 



^-^/-'(0-r(l + l/y) 
= 1. (24) 

Therefore, taking 5 = e, the entropy for the stretched-exponential distribution is seen to 
satisfy the Lesche stability condition. 

In the particular case when y— >l-0, S^^ipl converges to the Boltzmann-Gibbs- 

T— 1 W 

Shannon entropy, 'S'bg^L/?] = Pflnp., as it should do. Thus, as a byproduct, 
stability of the Boltzmann-Gibbs-Shannon entropy shown in Ref. [3] is ascertained. 

2. Quantum-group entropy 

The quantum-group entropy is given by 



q-q 
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This quantity has been introduced in Ref. [6] and has been appHed to generaUzed 
statistical-mechanical study of ^-deformed oscillators. In this expression, q is assumed to 
be positive. Since Sqq[p\ is symmetric under interchange, ^ the range of q can be 

reduced to (0, 1). This quantity also converges to the Boltzmann-Gibbs Shannon entropy 
in the limit ^ — > 1. 

The function, /(O, defined on {t^^, t^) = (-1, oo) associated with the quantum-group 
entropy is implicitly given as the inverse function of 



For this function, C in Eq. (21) is still an indeterminate form. Accordingly 
B(\\p- p'\\y,W) in Eq. (18) is evaluated directly as follows: 




(26) 



q-q 





rl/W , 



(\\p-p'\\, /Wy -(\\p-p'\\, /Wy -(q-q-')(l/W)\\p-p'\\ 



^(\\p-p'\\,y 



(27) 
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Therefore, taking d = e we see that the quantum-group entropy is stable. 
3. K" -entropy 

The Jc -entropy has been introduced in Ref. [7] and has been applied to systems 
described by statistical distributions having a power-law asymptotic behavior. In Ref. [7], 
the following one-parameter generalizations of the ordinary exponential and logarithmic 
functions have been proposed: 



from which the ordinary exponential and logarithmic functions are respectively 
reproduced in the limit K" — > +0. K should be in the range (-1, 1). On the other hand, 
exp(pj(0 is defined for t s (-oo, oo) and ln(^,(0 for t s (0, oo). Both of these functions are 
symmetric under interchange K ^-K. [For exp(^j(r), this interchange is combined with 
the reversal of t.] Therefore, the range of K can be taken to be (0, 1). 
It is a simple task to verify that if we choose 




(28) 



-K 



2k 



(29) 



/(0 = exp(^,(-0, 



(30) 
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f-'(t) = -In. At) fovte(t^,t^) = iO,oo) (31) 



then we obtain the K" -entropy given by 



f 

SAP^ = -L [c-. [(py-" - p] + [{pd'^" - p,]}, (32) 

(=1 



" 2 



- + (33) 



Substituting Eq. (31) into Eq. (18), we find 

B{\\p-p\\„W)^{^\p-p\\,]'~'' {W^oo). (33) 

Therefore, setting 5 = £^"-^~'^\ we conclude that the K" -entropy is stable [10]. It is worth 
mentioning that the /f -entropy becomes reduced to the Boltzmann-Gibbs-Shannon 
entropy in the limit K" — > +0 . 

V. CONCLUDING REMARKS 

We have discussed the generalized entropy optimized by a given arbitrary distribution 
under the constraints on normalization of the distribution and the ordinary expectation 
value of a physical random quantity. We have examined its Lesche stability property and 
have derived a general condition, under which the generalized entropy becomes stable. 



13 



We have also discussed some examples of entropic measures known in the literature and 
have shown their stabilities. 

In recent years, much attention has been paid to the Tsallis entropy [13], which has 
been employed for generalizing Boltzmann-Gibbs statistical mechanics for nonextensive 
systems [14-16]. It has been shown in Ref. [4] that the Tsallis entropy is stable. 

There are also unstable entropies. Examples are the Renyi entropy [17] and the so- 
called normalized Tsallis entropy [18,19], whose instabilities have been shown in Ref. [3] 
and Ref. [4], respectively. Quite interestingly, these quantities cannot be expressed in the 
form in Eq. (1) and are not concave if their entropic indices are larger than unity. 

Finally, we point out that mathematically the Lesche stability property is equivalent to 
uniform continuity of the functional under consideration. The problem of continuity itself 
has recently been studied in Refs. [20,21], where the Boltzmann-Gibbs-Shannon entropy 
is shown not to be continuous for infinite microscopic states. 

Acknowledgment 

S.A. was supported in part by the Grant-in- Aid for Scientific Research of Japan 
Society for the Promotion of Science. 

References 

[1] E.T. Jaynes: Papers on Probability, Statistics and Statistical Physics, edited by 
R. D. Rosenkrantz (Kluwer Academic Publishers, Dortrecht, 1989). 



14 



[2] S. Abe, J. Phys. A 36, 8733 (2003). 

[3] B. Lesche, J. Stat. Phys. 27, 419 (1982). 

[4] S. Abe, Phys. Rev. E 66, 046134 (2002). 

[5] C. Anteneodo and A. R. Plastino, J. Phys. A 32, 1089 (1999). 

[6] S. Abe, Phys. Lett. A 224, 326 (1997); ibid. 244, 229 (1998). 

[7] G. Kaniadakis, Physica A 296, 405 (2001); Phys. Rev. E 66, 056125 (2002). 

[8] G. Kaniadakis and A. M. Scarfone, Physica A 305, 69 (2001). 

[9] J. Naudts, Physica A 316, 323 (2002). 
[10] G. Kaniadakis and A. M. Scarfone, e-print cond-mat/03 10728, to appear in a 

special issue of Physica A (2004). 
[11] In Ref. [2], it was mentioned that the functional in Eq. (6) is not concave, in general. 
However, under the differentiability condition, the quantity in Eq. (6) in fact turns 
out to be a concave functional, as shown in Ref. [2]. 
[12] J. Naudts, e-print math-ph/0208038. 
[13] C. Tsallis, J. Stat. Phys. 52, 479 (1988). 

[14] Nonextensive Statistical Mechanics and Its Applications, edited by S. Abe 

and Y. Okamoto (Springer- Verlag, Heidelberg, 2001). 
[15] Special issue of Physica A 305, Nos. 1/2 (2002), edited by G. Kaniadakis, 

M. Lissia, and A. Rapisarda. 
[16] Nonextensive Entropy — Interdisciplinary Applications, edited by M. Gell-Mann 

and C. Tsallis (Oxford University Press, Oxford, 2004). 
[17] A. Renyi, Probability Theory (North-Holland, Amsterdam, 1970). 
[18] P. T. Landsberg and V. Vedral, Phys. Lett. A 247, 21 1 (1998). 



15 



[19] A. K. Rajagopal and S. Abe, Phys. Rev. Lett. 83, 1711 (1999). 
[20] P. Harremoes and F. Tops0e, Entropy 3, 191 (2001). 

[21] P. Harremoes, to appear in a special volume of the Janos Bolyai Mathematical 
Society (2004). 



16 



